Recently, the classical test theory and the Item Response Theory are the two major theories used in designing a test and processing the result of the test. A general introduction to each of the test theories is given in Chapters 6 and 7 of “Statistical Methods in Psychology” edited by Hiroshi Watabe (2002 by Fukumura Shuppan Inc.), respectively. Tests to which test theories are applied includes not only an achievement test, but also a psychological personality test, a clinical examination, etc. In the following description of the present invention, however, instead of presenting an abstract explanation covering these various fields, the inventors try to provide a more specific explanation in relation to achievement tests including foreign language tests in order for the present invention to be understood more readily. An “item” in the Item Response Theory corresponds to a problem in an achievement test. Therefore, in the specific explanation to be given in this specification, well-known terms in an achievement test will be used. For example, the terms “problem” and “testee”, which are common in the field of achievement tests, will be used instead of more generic terms “item” and “subject”, respectively. However, the present invention is not limited to an achievement test, but can be applied to the above-mentioned various fields to which test theories are applied.
A conclusion reached through the classical test theory, which is widely used, depends on the statistical distribution of the features of a testee. Therefore, when the classical test theory is used, it is not appropriate to compare average marks and deviations obtained in a plurality of tests conducted for a group of testees with different features.
The Item Response Theory has been proposed to overcome the shortcomings of the classical test theory. At present, there are a number of achievement tests which are designed and whose result are processed based on the Item Response Theory. For example, the following description can be seen in “Introduction to the Item Response Theory” by Hideki Toyoda (2002 by Asakura Shoten), on page 16. “TOEFL Examination, a famous language test, is conducted several times a year all over the world. Since one subject may sit for the test more than once, the same test cannot be presented as a group of the same items. Therefore, the average values and the passing rates are different among the tests. Since the ability in English varies in accordance with the areas, the distribution of the characteristic values is also different depending on the area. Therefore, it is impossible to compare the test results using the deviations, normal marks, and empirical distribution functions. - - - The marks of TOEFL Examination (for example, 500, 650, etc.) are used in determining the ability of studying abroad regardless of where and when the test was conducted, which problems are solved, etc. That is, although different testees take the examinations consisting of different items on different dates at different places, these testees can be treated with a unified manner. The Item Response Theory is a mathematical model for designing a system that allows such tests to be carried out continuously.”
In addition, Japanese Patent Publication No. 2002-006734 discloses a test method and system, wherein problems are pre-arranged in a tree structure, the problems are sequentially presented along a path in the tree structure depending on whether the testee has given the correct answer or the wrong answer to the previous problem, and the ability of a testee is estimated not only based on the number of correct answers, but also based on which path has been followed to the final point in the tree structure. The Item Response Theory is referred to in this published patent application.
However, in item response models based on the conventional theory, it has not been easy to handle a problem other than multiple-choice problems the answers to which can be marked as binary values of 1 or 0. For example, in such models, it is not easy to handle a problem the answer to which inevitably has to be marked as a partial score between 1 and 0. Although graded response models have been proposed to handle multi-valued results, such prior art methods suffer from the shortcomings to be described later. This is why essay or oral tests which cannot be appropriately marked without allowing partial scores are not typically included in a computer adaptive test (CAT). The invention disclosed in the above-mentioned Japanese Patent Publication No. 2002-006734 is an example wherein Item Response Theory is explicitly employed, and wherein the problems to be presented are pre-arranged in a tree structure and when a testee correctly answers a question, the lower right problem will be given as the next problem while when a testee cannot correctly answer the question, the lower left problem will be given as the next problem. Thus, the solution is treated in binary, that is, based on a correct or false answer.